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Abstract 

Background: The aim of this study was to explore if a course consisting of lectures combined with simulator 
training in coronary angiography (CA) could accelerate the early learning curve when performing CA on patients. 
Knowledge in performing CA is included in the curriculum for the general cardiologist. The method, according to 
American College of Cardiology and European Society of Cardiology guidelines, for this training is not well defined 
but simulator training is proposed to be an option. However, the transfer effect from a CA simulator to 
performance in real world cath lab is not validated. 

Methods: Fifty-four residents without practical skills in CA completed the course and 12 continued to training in 
invasive cardiology. These residents were tracked in the Swedish Coronary Angiography and Angioplasty Registry 
and compared to a control group of 46 novel operators for evaluation of performance metrics. A total of 4472 CAs 
were analyzed. 

Results: Course participants demonstrated no consistent acceleration in the early learning curve in real world cath 
lab. They had longer fluoroscopy time compared to controls (median 360 seconds (IQR 245-557) vs. 289 seconds 
(IQR 179-468), p < 0.001). Safety measures also indicated more complications appearing at the ward, in particular 
when using the femoral approach (6.25% vs. 2.53%, p < 0.001). 

Conclusions: Since the results of this retrospective non-randomized study were negative, the role of a structured 
course including simulator training for skills acquisition in CA is still uncertain. Randomized transfer studies are 
warranted to justify further use of simulators for training in CA 

Keywords: Learning curve. Simulator, Performance, Coronary angiography 



Background 

According to national and international guidelines for 
education and training of the general cardiologist, cor- 
onary angiography (CA) experience is of high priority. 
However, the rationale behind these curriculums is 
vague since training goals for cardiologist trainees often 
are built on recommendations without scientific support. 
A log-book is a common way to register number and 
type of procedures performed but is limited by volume 
instead of quality. European Society of Cardiology (ESC) 
recommendations for the general cardiologist are to as- 
sist or perform 300 CAs and to interpret 1000 investiga- 
tions [1]. There is also a statement that procedural skills 
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simulators might play an important role in training inva- 
sive procedures but recommendations for how to ac- 
complish this is lacking. The recommendation from the 
American College of Cardiology (ACC) is to participate 
in CA of at least 100 patients and for the trainee who 
plans to perform independent diagnostic cardiac cathe- 
terizations a minimum of 200 procedures with primary 
hands-on responsibilities should be performed [2]. 

Simulator training in CA is not well validated and 
transfer studies are lacking. Experts usually demonstrate 
a higher proficiency level in different simulator tasks 
and procedures as reported in several construct valid- 
ation studies and the same is true for some endovascular 
procedures [3-7]. Most randomized controlled trials 
(RCTs) evaluating procedural skills achieved in virtual 
reality (VR) transferred to the operating room (OR) have 
explored surgical procedures such as laparoscopy and 
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flexible endoscopy [8-10]. A common feature of these 
RCTs are that they are few and conducted with a rather 
limited number of participants as concluded in several 
systematic reviews [11-13]. None of the RCTs have had 
patient outcome as an end-point which should be the ul- 
timate criterion for the quality of the training programs 
using procedural skills simulators. A review by Lynagh 
et al. aimed for evaluating the effectiveness of medical 
skill laboratories or procedural skills simulators [14]. 
Twelve of the included studies assessed the transfer of 
simulator performance to clinical skills performance on 
real patients but none on endovascular procedures. The 
conclusion drawn from this review was that medical 
skills laboratories do lead to improvement compared to 
standard training when transferred to real life but that 
there is a lack of well-designed trials. A Cochrane review 
by Walsh et al. concluded that there is insufficient evi- 
dence to advise for or against the use of VR training, this 
time regarding gastrointestinal endoscopy [15]. A recent 
review and meta-analysis including all simulator training 
environments from practical skills training to team 
training has been conducted as an attempt to justify 
Simulation-Based Medical Education (SBME) as a con- 
cept for the future of medical training [16]. This meta- 
analysis of technology-enhanced simulation by Cook 
et al. identified 609 studies of which 137 were random- 
ized [16]. Only 10 studies explored the training effect in 
endovascular procedures. The overall conclusion was 
that knowledge, time skills and general behavior were fa- 
vored by simulator training but no sub-group analysis 
was performed regarding procedural skills in endovascu- 
lar procedures. When exploring the transfer effect from 
endovascular simulators to real patients a review article 
by Tsang et al. included three studies on carotid stenting 
and four on peripheral vascular angioplasty [17]. Only 
one of the RCT's showed transferability from VR to OR 
and that was in peripheral vascular intervention [18]. In 
a recently published review about the future of simula- 
tion technologies for complex cardiovascular procedures 
references were made to several VR validation studies in 
endovascular procedures; however all of those studies 
used animal models for vahdation [19] and to our know- 
ledge, no study has evaluated the transfer of CA skills 
from VR to OR in humans. 

The aim of this study was to evaluate if a structured 
training program including simulator training could im- 
prove the early learning curve for trainees in CA and 
thus make the learning process safer for the patient 
(transfer validity). 

Methods 

Course 

The course was founded and initiated by the authors in 
2006 and only minor changes have been made over the 



years. The course was recommended by the Swedish So- 
ciety of Cardiology and the Swedish Heart Association. 
At all course events residents had access to two simula- 
tors and three instructors. Each course was limited to 
six participants in order to keep a high exposure to the 
simulators. Three instructors, all experienced invasive 
cardiologists, were responsible for proctoring and lec- 
tures. A total of six hours of dyad proctored simulator 
training and six hours of theoretical lectures were com- 
pleted during two days. Course participants aiming for 
certification in CA and living in proximity to the train- 
ing center had an opportunity to obtain further solo VR 
experience and to perform a practical examination on 
the simulator. A goal of obtaining certification was not 
compulsory for course participation. 

The goal of VR training was to obtain a safe behavior of 
the procedure completing CA with a small but sufficient 
amount of contrast used and accurate virtual C-arm angu- 
lations to project the coronary vessels in recommended 
views. Instructions of how to handle fluoroscopy, wires 
and catheters safely during CA were also given. In 
addition, femoral arterial puncture technique was prac- 
ticed on a dummy with "subcutaneous" arterial-like rub- 
ber tubes constructed to give pulsatile backflow of 
artificial blood when entering the vessel. Puncture tech- 
nique ad modum Seldinger was demonstrated by the tu- 
tors in a stepwise fashion to ensure that all parts were 
accomplished in a correct way. The course participants 
had subsequent dyad practice in arterial puncture tech- 
nique for an average of 90 minutes. 

The theoretical part of the course included lectures 
about the CA procedure regarding anatomy, pharmacol- 
ogy, complications, puncture technique, radiation safety 
and materials, in total six hours. A web-based theoretical 
course was offered as a complement to live lectures dur- 
ing the two last years. 

Study subjects 

The course participants were all senior residents in car- 
diology and in their second half of their four years of 
training. They were recruited from all geographical areas 
of Sweden to attend the course by advertisement in the 
journal of Swedish Society of Cardiology and by direct 
mail to all cardiology units and invasive centers in 
Sweden. During six years, 54 residents participated and 
completed the course at two different sites in Sweden. 
Twelve of the course participants progressed to become 
invasive cardiologists. Five of these participants had free 
optional additional training in the simulator to enable 
examination and certification. 

Simulator 

The two centers involved in the course had each access 
to one VR simulator (Mentice VIST'") on a dedicated 
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Figure 1 Median fluoroscopy time for course participants and controls representing the early learning curve. CAs = coronary 
angiographies. Values in median (IQR). 



center for simulation (Clinical training center, Karolinska 
University Hospital, Stockholm and Practicum, Skane 
University Hospital, Lund). During the course an add- 
itional identical simulator was borrowed from one com- 
pany involved in the device industry. Mentice VIST™, 
Gothenburg, Sweden is a VR simulator where you can 
practice coronary angiography in full scale, using real 
catheters and wires modified to fit the machine. The vir- 
tual femoral arterial access was premade. Potentially 
harmful parts in the investigation such as radiation, 
fluoroscopy and filming were simulated as well as con- 
trast injection. 

Study protocol 

The design of the study was a retrospective cohort study 
where the cohort were residents exposed to the course 
progressing to invasive cardiologists and the controls 
novel operators found in the Swedish Coronary Angiog- 
raphy and Angioplasty Registry (SCAAR). All hospitals 
in Sweden performing CA (n = 30) and interventions 
(n = 29) register all their procedures. The definition of 
a beginner was set to be an invasive cardiologist who 
started to perform CA between 2005 and Ql 2012 and 
had performed at least 80 CAs and at least 40 CAs an- 
nually. A total of 58 novel CA operators were identi- 
fied in Sweden during the observation period of seven 
years. Twenty percent attended the course. Cohort (n = 12) 
and controls (n = 46) were tracked in the SCAAR registry 
and met the inclusion criteria for beginners. No other prac- 
tical simulator-based courses in CA were held in Sweden 
during the observation time and the likelihood for controls 
to be simulator trained in CA was low. There were no gen- 
der differences between the two groups (16.6% vs. 17.3% 



females). Study metrics representing proficiency in CA 
have been previously described and these were compared 
between groups [20]. Complications during CA is associ- 
ated to proficiency and during training most often related 
to the access site with increased risk of bleeding when 
using the femoral approach. The number of complications 
was therefore analyzed also in relation to access site. 
Elapsed time from course completion to performing the 
first CA or previous CA experience might have an impact 
on the real life performance and was therefore also 
explored. 

Statistical analysis 

Data are presented as median and inter-quartUe range 
(IQR), mean ± SD or median (range) and numbers (%). De- 
scriptive summary statistics were used where appropriate. 



Table 1 Metrics course participants to control 



Metric 




Course - [46] 


Course -1- [1 2] 


p-value 


Contrast tota 




70 (55-90) 


70 (55-95) 


0.687 


Fluoro CA 1- 


0 


329 (212-498) 


430 (278-597) 


0.001 


Fluoro CA 1 1 


-20 


322 (203-486) 


420 (283-670) 


<0.001 


Fluoro CA 21 


-30 


289 (179-510) 


415 (297-606) 


<0.001 


Fluoro CA 31 


-40 


287 (1 79-480) 


364 (252-522) 


<0.001 


Fluoro CA 41-50 


267 (1 74-435) 


317 (228-466) 


0.026 


Fluoro CA 51 


-60 


264 (1 70-445) 


309 (230-476) 


0.018 


Fluoro CA 61 


-70 


254 (156-419) 


287 (197-402) 


0.049 


Fluoro CA 71 


-80 


267 (179-432) 


337 (212-627) 


<0.001 


Fluoro CA 1- 


30 


289 (1 79-468) 


360 (245-557) 


<0.001 



Metrics course participants to control. Median values in seconds, (IQR). [# case 
or control]. Fluoro = fluoroscopy. + indicates course participants. - indicates 
controls. Mann-Whitney L/test. CA = coronary angiography. 



Jensen ef al. BMC Medical Education 2014, 14:49 
http://www.biomedcentral.com/1472-6920/14/49 



Page 4 of 8 



Differences were tested with Mann- Whitney U-test or 
Chi-Square test. Kruskal-Wallis was used where appropri- 
ate. Analyses were performed using Statistica version 10, 
(StatSoft, Inc, Tulsa, OK, USA) 

Ethics 

All participants received written information about the pro- 
ject. The protocols and procedures were approved by the 
local ethics committee for human research at Karolinska 
Institutet and at Uppsala University refnr. 04-202/1. The 
studies were performed according to the declaration of 
Helsinki and good clinical practice. Informed consent was 
provided by all participating residents and consultants. 



Analyzing retrospective CA procedural data in the SCAAR 
registry was covered by a general approval from the ethics 
committee at Uppsala University. 

Results 

A total of 4472 CAs were analyzed in the SCAAR registry. 
In the metrics extracted from SCAAR and representing 
proficiency in cath lab, the trainees completing the course 
performed worse regarding fluoroscopy time compared to 
the controls which in turn demonstrated a typical learning 
curve showing reduction of fluoroscopy time over time 
(Figure 1) [20,21]. Course trainee fluoroscopy time was in 
median 360 seconds (IQR 245-557) vs. 289 seconds 
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Figure 2 Median fluoroscopy time course participants and controls representing the early learning curve a) without delay and 
b) with delay from course to first CA. CAs = coronary angiographies. Values in median (IQR). 
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(IQR 179-468), p < 0.001, in non-course trainees (Table 1). 
Course participants demonstrated a less consistent im- 
provement and the learning curves between the groups 
did not persistently cross during the observation time. 
The pattern was the same independent of prior experience 
or lapsed time between taking the course and performing 
the first CA (Figure 2a-b), (Table 2). No learning curve 
was demonstrated in the use of contrast and the groups 
used the same amounts (Figure 3). There were no differ- 
ences in the rate of complications at the cath lab, 0.57% 
vs. 0.92%, p = 0.421 but the course participants had more 
complications appearing at the ward, in particular, when 
using the femoral approach, 6.25% vs. 2.53%, p < 0.001 
(Table 3). The total time for the procedure is not regis- 
tered in SCAAR and could therefore not be evaluated. 

Discussion 

In this study, the simulator-based course did not result 
in an acceleration of the learning curve, instead course 
participants had longer fluoroscopy time than controls, 
moreover time from course event or previous CA ex- 
perience did not affect the performance metrics. The 
number of complications in the course participants was 
elevated compared to the control group suggesting that 
a structured course including simulator training might 
have a negative impact on the learning process of CA. 

Transferability 

Simulation experience in medical procedures is regarded 
as the future for medical education and training. How- 
ever, convincing data of transferability from VR to OR in 
endovascular procedures have been shown only for a 
few procedures. Berry et al. showed a transferability 



Table 2 Baseline experience at course participation or 
delay to first CA 



Participant 


Course 


First CA 


Experience/time - angio 


1 


060914* 


080110 


69 weeks 


2 


060914 


090907 


155 weeks 


3 


071119* 


071023 


17 CA 


4 


08091 1 


080228 


27 CA 


5 


080911* 


110617 


144 weeks 


6 


081211 


090114 


5 weeks 


7 


081211 


090219 


9 weeks 


8 


081211 


090309 


12 weeks 


9 


091126 


090908 


31 CA 


10 


091126* 


090915 


33 CA 


11 


091126 


090212 


35 CA 


12 


100610* 


1 1 1 1 28 


77 weeks 



Baseline experience at course participation or delay to first CA. *marks 
participants with additional sim-training. CA = coronary angiography. 



from VR to a OR in a pig model of iliac vascular inter- 
vention and De Ponti et al. concluded that VR training 
in cardiac transseptal puncture (TSP) resulted in a 
shorter training time, a higher assessment score and 
fewer errors during TSP in patients [22,23]. No data 
have been published regarding VR transfer effect to CA. 

Aim and results 

In this study, our aim was to evaluate if a course in CA 
using simulators could accelerate the early learning 
curve in performing CA on real patients and if the pa- 
tient benefited from this preparatory VR training. The 
results demonstrated that using simulators as a learning 
tool to increase clinical skills is not convincingly obvious 
and that the training actually can impair the early learn- 
ing curve and result in worse outcome in patients. 

Only one previous study has reported an impaired per- 
formance in a VR trained group and that in a non- 
endovascular procedure [24]. In our study, residents taking 
a course in CA, including theoretical and practical training, 
actually performed worse in parameters previously demon- 
strated representing proficiency [20]. Course participants 
used longer fluoroscopy time and had more complications 
when using the femoral access. The initial learning curve 
was not altered whether they had a delay from the course 
or not to CA or had some CA experience when taking the 
course assuming that not even a practical course in close 
proximity to performing the procedure in real life had any 
positive effect on performance. 

Potential detrimental effect of VR training 

Unfortunately, this study could not demonstrate a trans- 
fer effect from VR to OR in this setting. One possible 
reason for that is that despite proctored VR training the 
actual simulator training was not structured in that 
sense that no benchmarked proficiency level was reached 
before performing CA on patients. Proficiency-based pre- 
test training has been a common denominator in VR stud- 
ies demonstrating transferability and proposed to be the 
paradigm shift in VR skills training [9,25]. However, by 
the time for these simulator-based CA courses the expert 
proficiency level in the simulator was not known. One 
might argue that a longer course or training period would 
have amplified the skills achieved in the simulator result- 
ing in a higher proficiency level in cath lab but on the 
other hand this would have been time-consuming for the 
trainee as well as for the proctors. A second hypothesis 
for the failure to show a transfer benefit from VR training 
to cath lab might be that the trainee becomes too self- 
confident after VR training thereby asking for less assist- 
ance with the actual CA procedure. In contrast the mental 
preparation for the procedure that simulator training of- 
fers might be beneficial. Mental imaginary of completing a 
stressful task has a potential to prepare the performer for 
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Figure 3 Median contrast delivery course participants and controls. CAs = coronary angiographies. Values in median (IQR). 



possible instrument handling difficulties and hazards and 
subsequent mental stress. However, a randomized study 
exploring whether mental imaginary of a surgical proced- 
ure could improve the performance failed to show any 
benefit in analogy with this preparatory simulator training 
[26]. The course attendees had longer fluoroscopy time, 
perhaps dependent of time spent on unnecessary handling 
of catheters something not dangerous to the VR patient. 
However, extended catheter handling in patients is corre- 
lated to catheter-related emboli and must be prevented 
[27]. Supervised stepwise training in cath lab like the old 
master-apprentice model with graded increased responsi- 
bility is perhaps as safe as simulator training and more cost 
efficient. Zendejas et al. [28] made an attempt to systemati- 
cal review cost as an outcome of SBME. Fifteen studies 
were indentified comparing simulation training to other in- 
structional modalities but none reported a formal cost- 
effectiveness analysis. Discussions promoting SMBE usually 
compare the cost of SBME to a hypothetical medical error 
thus saving money. However, in this study course partici- 
pants actually had more complications than the control 
group using the femoral approach indicating a higher cost. 



Reducing procedural complications 

The types of complications were not classified but bleed- 
ing associated to the puncture site is likely to be the 
major part. Puncture of the access site is not achievable 
in the VIST simulator instead the course participants re- 
ceived arterial puncture training on a dummy. This 
training modality is not validated in a transfer setting 
and might therefore be of no use or even harmful. To 
overcome this problem one might argue that the arterial 
puncture training simulator must consist of a more arter- 
ial tissue-like texture and be validated in a proficiency- 
guided randomized transfer environment. To optimize 
training conditions in a procedural skills simulator like 
VIST and to promote an ultimate training effect transfer- 
ring to real life some circumstances are likely to increase 
this effect: First, proficiency guided training, i.e. well- 
defined training goals in the simulator in different quality 
metrics known to affect patient outcome established by 
experienced CA operators. Second, quality and not time- 
dependent training should be performed to ensure a high 
lowest threshold for passing the training course. Finally, 
since access site complications are more common during 



Table 3 Complications during the first 80 procedures 

Course Lab comp Ward comp Fern lab comp Rad lab comp Fern ward comp Rad ward comp 

+ [12] 5/878(0.57) 38/878 (4.33) 3/528 (0.57) 2/350 (0.57) 33/528(6.25) 5/350 (1.43) 

-[46] 33/3594 (0.92) 67/3594(1.86) 19/1973 (0.96) 14/1620 (0.86) 50/1973 (2.53) 17/1620 (1.05) 

Total 38 105* 22 16 83* 22 



Complications during the first 80 procedures, (%). [participants]. * = p < 0.001 tested by Chi-Square. Comp = complication, Fern = femoral, Rad = radial. + indicates 
course participants. - indicates controls. 
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training, a validated arterial puncture procedural training 
facility, dummy or simulator, must be added to the course 
curriculum to enable a safe procedure from start to end. 

A randomized single-blinded VR transfer study in 
CA based on the above arguments is currently running 
in Stockholm and will be completed early 2014 and 
hopefully the future role for VR training in CA will be 
clearer. 

Study limitations 

The limitations in this study were several. It was not 
randomized and thereby not adjusted for potential con- 
founders such as poor performance of the course partici- 
pants. Course attendees progressing to invasive cardiology 
were few but corresponded to 20% of all novel operators 
in Sweden over the observation time of seven years. The 
course was short, only including 6 hours of simulator 
training, but the trainees were practicing in pairs known 
to increase the learning process and the sessions were 
proctored by experienced invasive cardiologists providing 
proximate feedback ensuring appropriate catheter behav- 
ior. Simulator training sessions were not guided to reach 
an expert level since this level was not known at the time 
of the courses. All participants did not advance straight to 
the cath lab doing CAs after completion of the course. 
Five of the course participants had some experience of 
performing CA and they all continued with CAs in direct 
proximity to the course without delay (Table 1). However, 
delay or experience did not seem to affect the early learn- 
ing curve (Figure 2a-b). 

The study is based on a well-validated registry, track- 
ing all coronary interventions in Sweden. However, this 
registry contains limited information regarding the oper- 
ator. For example, we do not know how much help or 
supervision the trainees had during their initial proce- 
dures in cath lab. Another limitation might be that only 
one procedural parameter was used for demonstrating 
competency but a previous registry study explored sev- 
eral such parameters and only fluoroscopy time demon- 
strated a true learning curve and an association to 
patient outcome [20]. Total time for a CA was not re- 
corded and the amount of radiation during a CA was 
not possible to measure since it is not comparable be- 
tween different sites because of different cath labs. These 
metrics is by all means important and might possibly 
represent proficiency level but could not be tested. How- 
ever the strength of this study is that it is multicenter 
excluding site bias and representing the real world 
situation. 

Conclusions 

Since the results of this retrospective non-randomized 
study were negative, the use of simulators is not neces- 
sarily associated with improved learning of CA. In this 



study, the concept of cognitive and practical training 
without a well-defined training goal resulted in an im- 
paired learning curve and worse performance in real life 
cath lab. Randomized transfer validation studies with 
well defined expert training goals are warranted to justify 
further use of simulators for CA training. 
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